Skip to content

Add Sessions for Automatic Conversation History Management #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 53 commits into
base: main
Choose a base branch
from

Conversation

knowsuchagency
Copy link

@knowsuchagency knowsuchagency commented May 24, 2025

Overview

Resolves #745

This PR introduces Sessions, a new core feature that automatically maintains conversation history across multiple agent runs, eliminating the need to manually handle .to_input_list() between turns.

Key Features

🧠 Automatic Memory Management

  • Zero-effort conversation continuity: Agents automatically remember previous context without manual state management
  • Session-based organization: Each conversation is isolated by unique session IDs
  • Seamless integration: Works with existing Runner.run(), Runner.run_sync(), and Runner.run_streamed() methods

🔌 Extensible Session Protocol

  • Library-agnostic design: Clean protocol interface allows any storage backend
  • Drop-in implementations: Easy integration with Redis, PostgreSQL, MongoDB, or any custom storage
  • Production-ready interface: Async-first design with proper error handling and type safety
  • Vendor flexibility: Library authors can provide their own Session implementations

💾 Built-in SQLite Implementation

  • In-memory SQLite: Perfect for temporary conversations during development
  • Persistent SQLite: File-based storage for conversations that survive application restarts
  • Thread-safe operations: Production-ready with connection pooling and proper concurrency handling

🔧 Simple API

# Before: Manual conversation management
result1 = await Runner.run(agent, "What's the weather?")
new_input = result1.to_input_list() + [{"role": "user", "content": "How about tomorrow?"}]
result2 = await Runner.run(agent, new_input)

# After: Automatic with Sessions
session = SQLiteSession("user_123")

result1 = await Runner.run(agent, "What's the weather?", session=session)
result2 = await Runner.run(agent, "How about tomorrow?", session=session)  # Remembers context automatically

What's Included

Core Session Protocol

  • Session Protocol: Clean, async interface that any storage backend can implement
  • Type-safe design: Full type hints and runtime validation
  • Standard operations: get_items(), add_items(), pop_item(), clear_session()
  • Extensibility-first: Designed for third-party implementations

Reference Implementation

  • SQLiteSession Class: Production-ready SQLite implementation
  • Automatic schema management: Creates tables and indexes automatically
  • Connection pooling: Thread-safe operations with proper resource management
  • Flexible storage: In-memory or persistent file-based databases

Runner Integration

  • New session parameter: Drop-in addition to existing Runner methods
  • Backward compatibility: Zero breaking changes to existing code
  • Automatic history management: Prepends conversation history before each run

Session Protocol for Library Authors

The Session protocol provides a clean interface for implementing custom storage backends:

from agents.memory import Session
from typing import List

class MyCustomSession:
    """Custom session implementation following the Session protocol."""

    def __init__(self, session_id: str):
        self.session_id = session_id
        # Your initialization here

    async def get_items(self, limit: int | None = None) -> List[dict]:
        """Retrieve conversation history for this session."""
        # Your implementation here
        pass

    async def add_items(self, items: List[dict]) -> None:
        """Store new items for this session."""
        # Your implementation here
        pass

    async def pop_item(self) -> dict | None:
        """Remove and return the most recent item from this session."""
        # Your implementation here
        pass

    async def clear_session(self) -> None:
        """Clear all items for this session."""
        # Your implementation here
        pass

# Works seamlessly with any custom implementation
result = await Runner.run(agent, "Hello", session=MyCustomSession("session_123"))

Example Third-Party Implementations

# Redis-based session (hypothetical library implementation)
from redis_sessions import RedisSession
session = RedisSession("user_123", redis_url="redis://localhost:6379")

# PostgreSQL-based session (hypothetical library implementation) 
from postgres_sessions import PostgreSQLSession
session = PostgreSQLSession("user_123", connection_string="postgresql://...")

# Cloud-based session (hypothetical library implementation)
from cloud_sessions import CloudSession
session = CloudSession("user_123", api_key="...", region="us-east-1")

# All work identically with the Runner
result = await Runner.run(agent, "Hello", session=session)

Benefits

For Application Developers

  • Reduces boilerplate: No more manual .to_input_list() management
  • Prevents memory leaks: Automatic cleanup and organized storage
  • Easier debugging: Clear conversation history tracking
  • Flexible storage: Choose the right backend for your needs

For Library Authors

  • Clean integration: Simple protocol to implement for any storage backend
  • Type safety: Full type hints and runtime validation
  • Async-first: Modern async/await design throughout
  • Documentation: Comprehensive examples and API reference

For Applications

  • Better user experience: Seamless conversation continuity
  • Scalable architecture: Support for multiple concurrent conversations
  • Flexible deployment: In-memory for development, production storage for scale
  • Multi-agent support: Same conversation history can be shared across different agents

Usage Examples

Basic Usage with SQLiteSession

from agents import Agent, Runner, SQLiteSession

agent = Agent(name="Assistant", instructions="Reply concisely.")
session = SQLiteSession("conversation_123")

# Conversation flows naturally
await Runner.run(agent, "Hi, I'm planning a trip to Japan", session=session)
await Runner.run(agent, "What's the best time to visit?", session=session)
await Runner.run(agent, "How about cherry blossom season?", session=session)

Multiple Sessions with Isolation

# Different users get separate conversation histories
session_alice = SQLiteSession("user_alice")
session_bob = SQLiteSession("user_bob")

# Completely isolated conversations
await Runner.run(agent, "I like pizza", session=session_alice)
await Runner.run(agent, "I like sushi", session=session_bob)

Persistent vs In-Memory Storage

# In-memory database (lost when process ends)
session = SQLiteSession("user_123")

# Persistent file-based database
session = SQLiteSession("user_123", "conversations.db")

Session Management Operations

session = SQLiteSession("user_123")

# Get all items in a session
items = await session.get_items()

# Add new items to a session
new_items = [
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi there!"}
]
await session.add_items(new_items)

# Remove and return the most recent item (useful for corrections)
last_item = await session.pop_item()

# Clear all items from a session
await session.clear_session()

Message Correction Pattern

# User wants to correct their last question
user_message = await session.pop_item()  # Remove user's question
assistant_message = await session.pop_item()  # Remove agent's response

# Ask a corrected question
result = await Runner.run(
    agent,
    "What's 2 + 3?",  # Corrected question
    session=session
)

Technical Details

Session Protocol Design

  • Async-first: All operations are async for non-blocking I/O
  • Type-safe: Full type hints with runtime validation
  • Error handling: Graceful degradation and detailed error messages
  • Resource management: Proper cleanup and connection handling

SQLiteSession Implementation

  • Thread-safe operations with connection pooling
  • Automatic schema management with proper indexing
  • JSON serialization for message storage
  • Memory-efficient conversation retrieval and storage
  • Cross-platform compatibility

Breaking Changes

None. This is a purely additive feature that doesn't affect existing functionality.

Documentation

  • Updated core concepts in docs/index.md to highlight Sessions as a key primitive
  • New comprehensive guide at docs/sessions.md with protocol implementation examples
  • Enhanced docs/running_agents.md with automatic vs manual conversation management
  • Full API reference integration via docs/ref/memory.md
  • Implementation guide for library authors

Sessions represent a significant architectural improvement for building conversational AI applications with the Agents SDK. The extensible Session protocol enables the ecosystem to provide specialized storage backends while maintaining a consistent, simple API for application developers.

## Summary
- Introduced `SessionMemory` and `SQLiteSessionMemory` classes for automatic conversation history management.
- Updated `Agent` class to support session memory configuration.
- Enhanced `Runner` class to handle input preparation and result saving with session memory.
- Added example demonstrating session memory usage.
- Implemented tests for session memory functionality.

## Testing
- `make format`
- `make lint`
- `make mypy`
- `make tests`
- Added a check to raise a ValueError if `session_id` is not provided when session memory is enabled.
- Updated the `SessionMemory` class to use a Protocol instead of an abstract base class, simplifying the implementation.
- Modified tests to ensure an exception is raised when attempting to run with memory enabled but no session_id is provided.
- Introduced a section on creating custom memory implementations following the `SessionMemory` protocol.
- Added code examples demonstrating how to implement and use a custom memory class.
- Highlighted the requirement for `session_id` when session memory is enabled, with examples illustrating correct usage.
- Updated the Runner class to ensure that when memory=True, a single instance of SQLiteSessionMemory is created and reused across runs.
- Added a test to verify that the same memory instance is returned for multiple calls when memory is enabled.
- Ensured the agent stores the memory instance for consistency.
- Updated README and example scripts to utilize `SQLiteSessionMemory` explicitly instead of using a boolean flag for memory.
- Modified `RunConfig` to accept a memory instance directly, enhancing clarity and flexibility in session management.
- Adjusted tests to reflect the new memory handling approach, ensuring consistent behavior across different configurations.
- Updated `mkdocs.yml` to include `session_memory.md` in the documentation.
- Enhanced `index.md` to highlight the new **Session Memory** feature for automatic conversation history management.
- Modified `running_agents.md` to include details about the `memory` and `session_id` parameters in `RunConfig`.
- Added comprehensive documentation for session memory functionality in the new `session_memory.md` file, including usage examples and best practices.
- Included `memory.md` in the documentation by updating `mkdocs.yml`.
- Corrected links in `session_memory.md` to point to the appropriate memory classes.
- Created a new `memory.md` file detailing the `SessionMemory` and `SQLiteSessionMemory` classes.
…essions and messages

- Updated the constructor to accept `sessions_table` and `messages_table` parameters, allowing users to specify custom table names.
- Modified SQL queries to utilize the provided table names, ensuring flexibility in database schema.
- Adjusted index creation and deletion queries to reflect the new table name parameters.
@knowsuchagency knowsuchagency marked this pull request as draft May 24, 2025 06:35
- Implemented the `pop_message` method to remove and return the most recent message from a session.
- Updated the `SessionMemory` protocol to include the new method signature.
- Enhanced documentation in `session_memory.md` with examples demonstrating the usage of `pop_message`.
- Added tests to verify the functionality of `pop_message`, including edge cases for empty sessions and multiple sessions.
- Converted synchronous database operations in `get_messages`, `add_messages`, `pop_message`, and `clear_session` methods to asynchronous using `asyncio.to_thread`.
- Improved performance and responsiveness of the session memory handling by allowing non-blocking database interactions.
- Implemented a check in the Runner class to raise a ValueError if a session_id is provided without enabling memory in the RunConfig.
- Updated tests to verify that the appropriate exception is raised when session_id is used without memory.
- Updated README, session_memory.md, and example scripts to remove the use of RunConfig for session memory configuration, directly passing memory and session_id parameters to the Runner.run method.
- Enhanced clarity in documentation regarding the requirement of session_id when memory is enabled.
- Adjusted tests to reflect the new approach, ensuring consistent behavior across different configurations.
- Introduced a new method `_init_db_for_connection` to handle database schema initialization for a specific connection.
- Updated the `_init_db` method to call the new method, improving clarity and separation of concerns.
- Added a comment to indicate the initialization of the database schema for the connection.
- Deleted the `_init_db` method as database schema initialization is now handled in `_init_db_for_connection`.
- This change simplifies the class structure and improves clarity in the database connection management.
- Introduced a shared connection for in-memory databases to avoid thread isolation, improving concurrency.
- Implemented a locking mechanism for database operations to ensure thread safety, regardless of the database type.
- Updated the `_get_connection`, `_add_messages_sync`, `_pop_message_sync`, and `_clear_session_sync` methods to utilize the new locking and connection management logic.
- Added logic to initialize the database schema for file databases during connection setup.
- Ensured that the schema is only initialized once, improving efficiency and clarity in connection management.
- Enhanced the error handling mechanism in the Runner class to ensure that exceptions during setup result in a completion sentinel being placed in the event queue.
- Streamlined the input preparation process by consolidating the logic for handling session memory and updating the streamed result.
- Improved clarity and maintainability of the code by restructuring the try-except blocks and ensuring proper resource management for spans and traces.
- Removed unused imports and streamlined the import statements for clarity and maintainability.
- This change enhances the readability of the test file by focusing on the necessary components.
@knowsuchagency knowsuchagency marked this pull request as ready for review May 24, 2025 21:37
…clarity

- Replaced `SQLiteSessionMemory` with `SQLiteSession` in the codebase, streamlining session management.
- Updated documentation and examples to reflect the new session handling approach, removing the need for `session_id` when using sessions.
- Enhanced the `Session` protocol to better define session behavior and improve consistency across implementations.
- Adjusted tests to ensure compatibility with the new session structure, maintaining functionality across various scenarios.
- Renamed `session_memory.md` to `session.md` for clarity and consistency.
- Updated links in `running_agents.md` to reflect the new documentation filename.
- Added comprehensive documentation for session memory functionality, including usage examples and API reference.
- Removed references to `SessionMemory` and `SQLiteSessionMemory` from the codebase to streamline session management.
- Changed all references from "Session Memory" to "Sessions" in README, documentation, and example files for consistency.
- Updated descriptions to clarify the functionality of Sessions in managing conversation history across agent runs.
- Renamed instances of "session.md" to "sessions.md" in mkdocs.yml and running_agents.md for consistency.
- Added new sessions.md file detailing the functionality and usage of session memory in the Agents SDK, including examples and API reference.
- Updated the `get_messages` method in the `Session` and `SQLiteSession` classes to accept an optional `amount` parameter, allowing retrieval of the latest N messages or all messages if not specified.
- Added a demonstration in `session_example.py` to showcase the new functionality for fetching the latest messages.
- Implemented tests in `test_session.py` to verify the behavior of the `get_messages` method with various amounts, ensuring correct message retrieval.
…tions" and clarify session implementation details. Adjusted section headers and descriptions for consistency with recent documentation updates.
@knowsuchagency knowsuchagency changed the title Add Session Memory for Automatic Conversation History Management Add Sessions for Automatic Conversation History Management May 25, 2025
@sibblegp
Copy link

Why aren't sessions managed on OpenAI's API side with just a session_id? That would be so much simpler than having us manage them on the client side.

This is a useless PR. I just dump .to_input_list() into Firebase. Then I fetch it. Literally two lines of code.

Sure, if you want to use SQLLite, this is great, but no way would I take the time to implement a custom version when I can do the same thing in 2 lines of code.

@knowsuchagency
Copy link
Author

The point of the PR is to provide the interface for session management services. SQLite is provided as a default implementation for convenience.

@sibblegp
Copy link

@knowsuchagency It just seems a conversation_id we could pass along with Runner which would retrieve context and previous messages from OpenAI so we don't have to send it every time would be easier. Not sure if your API has this capability yet though.

@knowsuchagency
Copy link
Author

The inspiration is Google's ADK sessions where they provide an InMemorySessionService and a VertexAiSessionService.

The goal is to provide scaffolding for OpenAI (and others) to provide the same functionality in the most intuitive way possible.

@seratch seratch added enhancement New feature or request feature:core labels Jun 25, 2025
@sibblegp
Copy link

Nice work, but honestly this could be achieved so easily if the OpenAI API provided something as simple as "session_id" and managed it all on their end.

I already handle this and all I do is serialize .to_result_list() into and out of firebase. It's super simple.

@rm-openai
Copy link
Collaborator

@knowsuchagency sorry for the long delay and thanks fore the discussion. I think this should be good to merge. Going to give it a thorough review right now - if you can fix the lint, will merge today. Thanks for the idea+implementation!

Comment on lines 665 to 666
if streamed_result.trace:
streamed_result.trace.start(mark_as_current=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason you moved this here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there was a regression in the test suite that made it necessary, but I don't remember what the specific issue was

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm ok do you think you could move it back and see if necessary?

Comment on lines 629 to 632
current_agent = starting_agent
current_turn = 0
should_run_agent_start_hooks = True
tool_use_tracker = AgentToolUseTracker()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possible to reduce the number of moves/unrelated changes? Would be nice to preserve blame and make code review easier

session_id: str

@abstractmethod
async def get_messages(self, limit: int | None = None) -> list[TResponseInputItem]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would limit be passed? Who decides what the limit is?

Also, thoughts on changing this to get_items? Messages is a super overloaded term

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main benefit is when you need to render only the last N items in the conversation. But I could also see it being useful for managing the overall size of the context windows i.e. having a sliding window of messages

### Changes
- Renamed methods in `MyCustomSession`, `SQLiteSession`, and related classes from `get_messages`, `add_messages`, `pop_message` to `get_items`, `add_items`, `pop_item`.
- Updated documentation and examples to reflect the new method names and terminology.
- Adjusted tests to ensure they validate the new method names and functionality.

This change enhances clarity by standardizing the terminology used for session items.
@knowsuchagency
Copy link
Author

I was able to lower the size of the diff in run.py in the last commit by 25 lines. Let me know if there are any other changes to make thanks @rm-openai !

Copy link
Collaborator

@rm-openai rm-openai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still seems like a ton of unrelated changes?

Comment on lines 665 to 666
if streamed_result.trace:
streamed_result.trace.start(mark_as_current=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm ok do you think you could move it back and see if necessary?

### Changes
- Simplified method calls in `AgentRunner` by using class methods directly.
- Consolidated input preparation and event queue updates for clarity.
- Improved formatting and reduced line breaks for better readability.

This refactor enhances the maintainability of the code by standardizing method usage and improving overall structure.
…stency

### Changes
- Removed the `session` field from `RunOptions` and its related comments to simplify the interface.
- Improved method calls by using class methods directly for better maintainability.

This refactor streamlines the code structure and improves the overall clarity of the `run.py` file.
…eparation in `AgentRunner`

### Changes
- Consolidated error handling to ensure `streamed_result.is_complete` is set even if an exception occurs.
- Improved the structure of input preparation and event queue updates for better readability and maintainability.

This refactor enhances the robustness of the `AgentRunner` class by ensuring proper completion signaling and clearer code flow.
### Changes
- Added `session` field to `RunOptions` for session management.
- Refactored `AgentRunner` methods to accept parameters via `kwargs`, improving flexibility and maintainability.

This update streamlines parameter handling and enhances the functionality of the `AgentRunner` class.
…rove clarity and maintainability

### Changes
- Moved the initialization of variables closer to their usage for better readability.
- Removed redundant exception handling to streamline the flow of the method.
- Ensured that the completion signaling for `streamed_result` is handled more clearly.

This update enhances the robustness and clarity of the `AgentRunner` class.
@knowsuchagency knowsuchagency requested a review from rm-openai June 28, 2025 00:27
@knowsuchagency
Copy link
Author

The changes in run.py should be a lot more straightforward now @rm-openai. The business logic is largely the same with minor edits to accommodate the new Session interface. LMK if you notice any other necessary edits, thanks!

@@ -143,3 +143,4 @@ cython_debug/
# PyPI configuration file
.pypirc
.aider*
.claude/settings.local.json

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knowsuchagency That seems not to be related to the PR.

LIMIT ? OFFSET ?
""",
(self.session_id, limit, offset),
)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knowsuchagency I think it will be cleaner and faster to use DESC and then just reverse result:

cursor = conn.execute(
    f"""
    SELECT message_data FROM {self.messages_table}
    WHERE session_id = ?
    ORDER BY created_at DESC
    LIMIT ?
    """,
    (self.session_id, limit),
)
rows = cursor.fetchall()

# To get them in chronological order (oldest to newest), reverse the list:
messages = list(reversed(rows))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature:core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Session Memory
9 participants